The technology described herein relates to a method of, and an apparatus for, processing a frame, in particular in data processing systems that generate output frames that are composited from or otherwise formed of one or more input frames.
Methods are known which reduce the number of memory accesses in a data processing system when processing an input frame to generate an output frame, e.g. for display on a display device. The input frame may be a graphics frame generated by a GPU of the system, a video frame received and decoded by a video engine, and/or a frame containing text, audio, etc. The input frames may be processed by a composition engine to composite two or more input frames into a single output frame, by a processor to perform sharpening, smoothing, etc., and/or by the video engine to encode a video frame before transmission. It is possible, for one or more stages of the processing to be performed on an input frame, to eliminate one or more of the read or write transactions between a memory and the module performing the processing. For example, this may be achieved by comparing a region (e.g. a processing tile) of the input frame to be read from or written to the memory with a region already read from (and e.g. stored in a buffer) or written to the memory, such that when the two regions are determined to be similar, the read/write transaction to be performed may be eliminated by reusing the region that has already been read/written.
These methods are particularly useful in cases where a series of input frames are similar to each other, for example having an unchanging background, such that regions of the unchanging background from one input frame may be reused for the subsequent input frames, and/or where an input frame has an area of plural regions that are similar to each other, for example the sky, the sea or a solid-colour background, a region of the area can be reused for other regions of the area.
However, the methods described above become less effective in cases where the input frames contain noise, such that comparison between two similar regions may not result in a match, and/or where two input frames being compared have the same content but have small differences such as horizontal shifting and/or slight difference in viewing angle.
An example include images captured using a digital camera. When an image is captured using a digital camera, the data is sent from the camera to the camera image signal processor (ISP). A number of processes may be performed by the ISP on the captured image, for example de-mosaicing, lens shading, defective pixel correction, etc. The resulting image and any image statistics, which are processed to determine a white balance, are written to memory. The image may then be fetched for further processes to be performed such as automatic white balancing, de-noise or sharpening, colour conversion, format conversion (to e.g. JPEG), etc., and the final processed image is output.
The processing of a captured image to generate an output image can therefore be extensive, and the number of memory accesses can be high. It is therefore desired to reduce the number of memory accesses for noisy content.
Another example, illustrated in FIG. 1, is stereoscopic 3D technologies, in which an output image 10 as perceived by a viewer comprises a pair of left 11 and right 12 images corresponding to a scene 13 as seen respectively by the left and right eyes of the viewer. In order for the viewer to perceive the scene in three dimensions, the scene in the left image 11 is shifted horizontally with respect to the scene in the right image 12 by the interocular distance (the distance between the eyes) 14, as illustrated in FIG. 1, to imitate the difference between the images seen by each eye.
The left and right images 11, 12 may be stored (and transmitted) in a left and right (side by side) or a top and bottom (one subsequent to the other) format, as illustrated in FIG. 2. The left and right images are fetched by a display controller and may be manipulated (processed) according to the output display technology. Stereoscopic 3D displays may display (or project) the left and right images 11, 12 as alternate frames (e.g. shutter-based or polarized RealD 3D technologies), as shown in FIG. 3 or as an interleaved frame (e.g. parallax barriers) 15, as shown in FIG. 4. Due to the difference in the viewing origin separated by the interocular distance, the viewing angles (and possibly the lighting conditions) of the left image and of the right image may have some, albeit small, differences.
It is therefore desired to reduce the number of memory accesses when processing frames, e.g. for display, for cases where the content of frame regions is expected to show some similarities (but without being exactly the same), so that bandwidth and power consumption in such situations may be reduced.
Like reference numerals are used for like features throughout the drawings, where appropriate.